The keyboard is an integral component to many computer systems and remains the primary means to insert (type) text. In the many years since its introduction, the keyboard has had some evolution such as the introduction of function keys, numeric keypads, and navigation keys. Nevertheless, these evolutionary enhancements have failed to overcome the keyboard's inability to efficiently navigate a graphical user interface (GUI) and invoke output actions therein. As a result, the keyboard remains relegated substantially to text insertion. Other input devices, such as the mouse, trackpad or touchscreen, are typically used to navigate the GUI and invoke output actions such as launching an application or playing a music file.
Though useful in navigating the GUI and invoking the output actions, the mouse and touchscreen input devices are unable to leverage many of the advantages inherent in the use of the keyboard. Firstly, the mouse and touchscreen input devices are spatially dependent whereas the keyboard is a discrete input device. This spatial dependence limits the speed by which an output action is invoked. Specifically, to invoke an output action with the mouse, a user first identifies a current (random) location of the mouse pointer on-screen. Next, the user identifies a path to an on-screen user interface (UI) element that invokes the desired output action. The user moves the mouse pointer from its existing location to the UI element where a click or double click of a mouse button invokes the output action. With a touchscreen interface, the user avoids the former step, but still performs the spatial identification of the UI element before moving the hand and fingers to invoke the output action with a touch gesture.
The speed by which a user can invoke the output action using the mouse or touchscreen input device is further limited when the UI element is obscured by the three dimensional nature of the GUI. For example, UI elements may be layered atop one another or hidden behind windows. Furthermore, the desired UI element may reside within various menu layers, graphical ribbons, folders, etc., that require the user to memorize and navigate through an object hierarchy before arriving at the desired UI element.
The spatial dependence of these input devices also requires the user to develop fine motor skills A mouse click or touchscreen gesture does not invoke an output action unless the mouse click/drag or the touchscreen gesture occurs within a specifically defined region of the UI element that invokes the output action, which may be quite small. Conversely, the discrete inputs of the keyboard may be used to invoke an output action using a particular key press combination. However, these key press combinations (using “flag” keys such as command, option, control, shift, function, alt, etc.) are difficult to remember and inconsistent across different applications and different output actions. For this reason, the discrete input capabilities of the keyboard have not been fully realized in the context of navigating the GUI and invoking output actions.
Secondly, the mouse and touchscreen input devices are unable to leverage muscle memory because of their spatial dependency. Each time a particular output action is to be invoked using the mouse, the user identifies the position of the UI element that invokes the particular output action and the user moves the mouse pointer to the UI element before being able to invoke the particular output action. Conversely, the keyboard is driven by muscle memory. Through sufficient repetition, users automatically recreate a set of actions without conscious effort. The greater the amount of repetition, the faster the user is able to recreate an action. For example, after a sufficient number of iterations, the commands for undo (e.g., ctrl-z or command-z) and cutting text (e.g., ctrl-x or command-x) become ingrained in the user's muscle memory. However, the muscle memory capabilities of the user have been relegated to a select number of output actions due to the sheer number of key combinations and inconsistencies in key combinations between different applications.
Thirdly, alternating between multiple input devices (i.e., the mouse and touchscreen input devices for navigation and the keyboard for text insertion) introduces another layer of spatial dependency. The physical movement of the hand between the mouse/touchscreen input device and the keyboard introduces delay when switching between GUI navigation and text insertion. This delay is further exacerbated when moving away from or back to the keyboard as the user loses and must reorient the user's hand to the precise home position on the keyboard.
Accordingly, there is a need to enhance software and hardware to better leverage the existing input devices that are used to interface with a computer system. There is a need to adapt the advantages of the keyboard to GUI navigation and output action invocation. There is also a need to enhance mouse and touchscreen functionality to improve GUI navigation and output action invocation.